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FIGs. [[2 is]] 2A - 2C are a block diagram of a preferred embodiment of a client side system, 
including speech capturing modules, partial speech processing modules, encoding modules, 
transmission modules, agent control modules, and answer/voice feedback modules that can be used 
in the aforementioned NLQS; 

FIG. [[2-2]] 2D is a block diagram of a preferred embodiment of a set of initialization routines and 
procedures used for the client side system of FIG. [[2]] 2A - 2C ; 

FIG. 3 is a block diagram of a preferred embodiment of a set of routines and procedures used for 
handling an iterated set of speech utterances on the client side system of FIG. [[2]] 2A - 2C . 
transmitting speech data for such utterances to a remote server, and receiving appropriate responses 
back from such server; 

FIG. 4 is a block diagram of a preferred embodiment of a set of initialization routines and 
procedures used for un-initializing the client side system of FIG. [[2]] 2A - 2C ; 

Please substitute the following for the text at page 26. 11. 20 - 32 : 

Client Side System 150 

The architecture of client-side system 150 of Natural Language Query System 100 is illustrated in 
greater detail in [[FIG. 2]] FIGs. 2A - 2C . Referring to FIG. 2A, the three main processes 
effectuated by Client System 150 are illustrated as follows: Initialization process 200A consisting of 
SRE 201, Communication 202 and Microsoft (MS) Agent 203 routines; at FIG. 2B an iterative 
process 200B consisting of two sub-routines: a) Receive User Speech 208 - made up of SRE 204 
and Communication 205; and b) Receive Answer from Server 207 - made up of MS Speak Agent 
206, Communication 209, Voice data file 210 and Text to Speech Engine 21 1. Finally, in FIG. 2C 
un-initialization process 200C is made up of three sub-routines: SRE 212, Communication 213, and 
MS Agent 214. Each of the above three processes are described in detail in the following 
paragraphs. It will be appreciated by those skilled in the art that the particular implementation for 
such processes and routines will vary from client platform to platform, so that in some 
environments such processes may be embodied in hard-coded routines executed by a dedicated 
DSP, while in others they may be embodied as software 



Preliminary Amendment for 9900 1 C1 



Please substitute the following for the text at page 27. 11. 4 — 



7: 



Initialization at Client System 150 

The initialization of the Client System 150 is illustrated in FIG. [[2-2]] 2D and is comprised generally 
of 3 separate initializing processes: client-side Speech Recognition Engine 220A, MS Agent 220B 
and Communication processes 220C. 

Please substitute the following for the text at page 27. 11. 22 - 25 : 
Initialization of MS Agent 220B 

The software code used to initialize and set up a MS Agent 220B is also illustrated in FIG. [[2-2]] 
2D . The MS Agent 220B routine is responsible for coordinating and handling the actions of the 
animated agent 157 (FIG.l). This initialization thus consists of the following steps: 

Please substitute the following for the text at page 30. 11. 30 - 32 : 
Initialization of Communication Link 160A 

The initialization of Communication Link 160A is shown with reference to process 220C FIG. [[2- 
2]] 2D. Referring to FIG. [[2-2]] 2D, this initialization consists of the following code components: 
Open 
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Uninitialization 

The un-initialization routines and processes are illustrated in FIG. 4. Three functional modules are 
used for un-initializing the primary components of the client side system 150; these include SRE 
270, Communications 271 and MS Agent 272 un-initializing routines. To un-initialize SRE 220A, 
memory that was allocated in the initialization phase is de-allocated by code 273 and objects created 
during such initialization phase are deleted by code 274. Similarly, as illustrated in FIG. 4, to un- 
initialize Communications module 220C the INTERNET connection previously established with 
the server is closed by code portion 275 of the Communication Un-initialization routine 271. Next 
the INTERNET session created at the time of initialization is also closed by routine 276. For the 
un-initialization of the MS Agent 220B, as illustrated in FIG. 4, MS Agent Un-initialization routine 
272 first releases the Commands Interface 227 using routine 277. This releases the commands added 
to the property sheet during loading of the agent character by routine 225. Next the Character 
Interface initialized by routine 226 is released by routine 278 and the Agent is unloaded at 279. The 
Sink Object Interface is then also released 280 followed by the release of the Property Sheet 
Interface 281. The Agent Notify Sink 282 then un-registers the Agent and finally the Agent Interface 
283 is released which releases all the resources allocated during initialization steps identified in FIG. 
[[2-2]] 2D. 
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Another embodiment of the invention is illustrated in FIG. 12. This web page illustrates a typical 
website that employs NLQS in a web-based learning environment. As illustrated in FIG. 12, the 
web page in browser 1200, is divided into two or more frames. A character 1210 in the likeness of 
an instructor is available on the screen and appears when the student initiates the query mode either 
by speaking the word "Help" into a microphone (FIG. 2B, 215) or by clicking on the link 'Click to 
Speak' (FIG. 12, 1280). Character 1210 would then prompt the student to select a course 1220 
from the drop down list 1230. If the user selects the course 'CPlusPlus', the character would then 
confirm verbally that the course "CPlusPlus" was selected. The character would then direct the 
student to make the next selection from the drop-down list 1250 that contains the selections for the 
chapters 1240 from which questions are available. Again, after the student makes the selection, the 
character 1210 confirms the selection by speaking. Next character 1210 prompts the student to 
select 'Section' 1260 of the chapter from which questions are available from the drop down list 
1270. Again, after the student makes the selection, character 1210 confirms the selection by 
articulating the 'Section' 1260 chosen. As a prompt to the student, a list of possible questions appear 
in the list box 1291. In addition, tips 1290 for using the system are displayed. Once the selections 
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